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* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation 
may not reflect the original precisely, 

2. **** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



CLAIMS 

[Claim(s)] 

[Claim 1]A face sensing device comprising: 

A point that a luminosity changes from ** to dark and ** when inside of 
a picture by which a person's face is photoed is investigated to a top and 
bottom direction of a picture. 

A means to detect a field across which it faces in that it changes from 
dark to •**. 

A means by which a shape facility of this field to this field and the 
luminosity feature in a field take out a field near a person's eyes as an 
eye candidate area. 

A means by which a shape facility of a described area to a described area 
and the luminosity feature in a field take out a field near a person's 
■ mouth as a mouth candidate area, A means to take out combination 
which combined one arbitrarily, without overlapping and has succeeded in 
arrangement appropriate for a person's face from two from the- above- 
mentioned eye candidate area, and the above-mentioned mouth 
candidate area, and to detect as a face-candidates field. 

[iIUaim-22AJace-s^nsing^^V4Ge-Gompr4si&g-; „ 

A point that a luminosity changes from dark to ** when inside of a 
picture by which a person's face is photoed is investigated to a top and 
bottom direction of a picture. 

A means to detect a field across which it faces in that it changes to 
^^H^^fc^HH^sH and ^fi^fi, 

A means by which a shape facility of this field to this field and the 
luminosity feature in a field take out a field near a person's eyes as an 
eye candidate area. 

A means by which a shape facility of a described area to a described area 
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and the luminosity feature in a field take out a field near a person's 
mouth as a mouth candidate area, A means to take out combination 
which combined one arbitrarily, without overlapping and has succeeded in 
arrangement appropriate for a person's face from two from the above- 
mentioned eye candidate area, and the above-mentioned mouth 
candidate area, and to detect as a face-candidates field. 

[Claim 3]A face sensing device comprising; 

A point that a luminosity changes from ** to dark and ** when inside of 
a picture by which a person's face is photoed is investigated to a top and 
bottom direction of a picture. 

A means to detect a field across which it faces in that it changes from 
dark to **. 

A means by which a shape facility of this field to this field and the 
luminosity feature in a field take out a field near a person's eyes as an 
eye candidate area. 

A means by which a shape facility of a described area to a described area 
and the luminosity feature in a field take out a field near a person's 
mouth as a mouth candidate area, A means which takes out combination 
which has succeeded in arrangement appropriate for [ combine one 
arbitrarily, without overlapping and ] a person's face from two from the 
above-mentioned eye candidate area, and the above-mentioned mouth 
candidate area as a face-candidates field, and a means to detect a 
person's face by comparing a standard face with the above-mentioned 
face-candidates field. 

[Claim 4] A face sensing device comprising: 

A point that a luminosity changes from ** tacitly when inside of a picture 
by which a person's face is photoed is investigated to a top and bottom 
direction of a picture. 

A means to detect a field across which it faces in that it changes to 

******** ant £ ** t 

A means by which a shape facility of this field to this field and the 
luminosity feature in a field take out a field near a person's eyes as an 
eye candidate area. 

A means by which a shape facility of a described area to a described area 
and the luminosity feature in a field take out a field near a person's 
mouth as a mouth candidate area, A means which takes out combination 
which has succeeded in arrangement appropriate for [ combine one 
arbitrarily, without overlapping and ] a person's face from two from the 
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above-mentioned eye candidate area, and the above-mentioned mouth 
candidate area as a face-candidates field, and a means to detect a 
person's face by comparing a standard face with the above-mentioned 
face-candidates field. 
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* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation 
may not reflect the original precisely. 

2. **** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



DETAILED DESCRIPTION 

[Detailed Description of the Invention] 
[0001] 

[Industrial Application]This invention is the art of detecting a person's 
face out of a picture, and is available to the supervising system of a 
building, or the visual system of a robot. 
[0002] 

[Description of the Prior Art]real time parallel cooperative recognition 
(Osamu Hasegawa.) of the person video for realization of a visual human 
interface with literature "natural feeling high as an example of a 
conventional system which detects a person's face from a portrait image 
Based on Kazuhiko Yokozawa, Mitsuru Ishizuka, and Institute of 
Electronics, Information and Communication Engineers paper magazine 
Vol.J77-D-II No.l Jan (1994)", it explains roughly. 
[0003]The algorithm used by this literature extracts the feature shown 
below as bottom up processing, when video is inputted. 
[0004]1. It can set on each point of 2500 points (50x50 points) among a 
picture among the motion vector 3. picture based on the inter-frame 
difference of 2500 points (50x50 points) among the average luminance 2. 

i^ictere^406-pcMrt^ 

which the difference between 2500 points (50x50 points) and a setting- 
out (background) picture set up into the big field 5. picture among the 
hair based on the ratio of an RGB value, and a beige field 4. picture — 
as for the feature extraction processings 3 and 4, setting out of a 
threshold is needed here. Since the scene which can be adapted if a fixed 
threshold is given decreases, by the result of the processing 1, the 
luminosity of the whole picture is distinguished and these thresholds are 
changed accommodative, 

[0005]In the processing 3, the threshold supposing the black hair which a 
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typical Japanese has, and flesh color is set up. A logical product is taken 
and the result of the processings 3 and 4 extracts the complexion range 
which does not exist in a setting-out picture. It uses, when each field is 
moved, it matches with information and the face which is moving is 
detected. 

[0006]The face model is built on the other hand, using the following 
conditions as top down processing. 

[0007] 1. The surface ratio of a hair field and a skin field is less than 
**30%, 

[0008]2. Only the quantity of the range which edge information (an eye, 
a nose, and a mouth) set up on the complexion range exists. 
[0009]3. The above-mentioned edge information has hot carried out 
uniform distribution, 

[0010J4. Edge information has gathered in the complexion range upper 
part comparatively (eyes). 

This processing is performed about the field considered that a face exists 
as a result of bottom up processing, and a detection result is used for the 
window setting for performing five under bottom up processing. 
[0011]As an example of conventional technology, one more. "Target 
image extraction, for face recognition, using thesub-space classification 
method (S. based on Akamatsu, T.Sasaki, H.Fukamachi, Y.Suenaga, and 
MVA'92 pp. 465-468(1992)', it explains roughly.) 
[0012]When the color picture containing a Japanese person face is 
inputted, it is supposed by observing this picture in a HSV color space 
and each YIQ color space that the following knowledge was acquired. 
[0013]- On the Q-axis of a YIQ color space, a lip has a high value as 
compared with the surroundings. 

[0014]- The skin field has a characteristic peak on the I-axis of a YIQ 
color space H axis top of a HSV color space. Then, after carrying out the 
convert colors of the color picture inputted by RGB to HSV and YIQ, 

perJormm.g^4hr^haM 

mentioned feature of a field detects a lip area. Eyes and a mouth make 
face candidates combination which has accomplished the arrangement 
appropriate for a face by making the hole in a skin field into an eye 
candidate. 

[00 15] What the obtained face candidates were mistaken in eyes and 
eyebrows, and was detected is contained. The position was normalized by 
eyes and a mouth, and the face candidates detected by drawing 2 were 
classified into six, and were shown, (a) is a picture acquired when a 
detection success is carried out, and (b) and (c) are the examples which 
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made a mistake that they were eyebrows in one side of eyes, 
respectively, (d) It is an example of the person to whom -(e) - (f) wore 
glasses, respectively, 

[0016]In this conventional technology, these six normal standard images 
are prepared, and it outputs noting that it is the face which detected the 
candidate nearest to (a) or (d). 
[0017] 

[Problem(s) to be Solved by the Invention]ln conventional technology, 
the threshold needed to be clearly given on the occasion of the 
complexion range extraction used as the core of the processing which 
detects a face. The threshold which was suitable for face detection by 
change of lighting conditions will change. Although the threshold is 
changed with the average luminance of the whole picture in the above- 
mentioned literature, it does not depend beige and the threshold for 
extracting flesh color, for example cannot necessarily be defined 
[ applicable ] with the average luminance of the whole picture. For this 
reason, the tougher algorithm is called for from change of lighting 
conditions. 
[0018] 

[Means for Solving the Problem]This invention is characterized by a face 
sensing device comprising the following. 

A point that a luminosity changes from ** to dark and ** when inside of 
a picture by which a person's face is photoed is investigated to a top and 
bottom direction of a picture. 

A means to detect a field across which it faces in that it changes from 
dark to #*, 

A means by which a shape facility of this field to this field and the 
luminosity feature in a field take out a field near a person's eyes as an 
eye candidate area. 

A means by which a shape facility of a described area to a described area 
^n4-t4e4uim.n©si-ty^ 
mouth as a mouth candidate area, A means to take out combination 
which combined one arbitrarily, without overlapping and has succeeded in 
arrangement appropriate for a person's face from two from the above- 
mentioned eye candidate area, and the above-mentioned mouth 
candidate area, and to detect as a face-candidates field. 

[0019]This invention is characterized by a face sensing device comprising 
the following. 

A point that a luminosity changes from dark to ** when inside of a 
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picture by which a person's face is photoed is investigated to a top and 
bottom direction of a picture. 

A means to detect a field across which it faces in that it changes to 
^^^^^^n^sH and ^^h. 

A means by which a shape facility of this field to this field and the 
luminosity feature in a field take out a field near a person's eyes as an 
eye candidate area, 

A means by which a shape facility of a described area to a described area 
and the luminosity feature in a field take out a field near a person's 
mouth as a mouth candidate area, A means to take out combination 
which combined one arbitrarily, without overlapping and has succeeded in 
arrangement appropriate for a person's face from two from the above- 
mentioned eye candidate area, and the above-mentioned mouth 
candidate area, and to detect as a face-candidates field. 

[0020]This invention is characterized by a face sensing device comprising 
the following. 

A point that a luminosity changes from ## to dark and ** when inside of 
a picture by which a person's face is photoed is investigated to a top and 
bottom direction of a picture. 

A means to detect a field across which it faces in that it changes from 
dark to **. 

A means by which a shape facility of this field to this field and the 
luminosity feature in a field take out a field near a person's eyes as an 
eye candidate area. 

A means by which a shape facility of a described area to a described area 
and the luminosity feature in a field take out a field near a person's 
mouth as a mouth candidate area, A means which takes out combination 
which has succeeded in arrangement appropriate for [ combine one 
arbitrarily, without overlapping and ] a person's face from two from the 

abov-e^mentianed^y^-carf 

candidate area as a face-candidates field, and a means to detect a 
person's face by comparing a standard face with the above-mentioned 
face-candidates field. 

[0021]This invention is characterized by a face sensing device comprising 
the following, 

A point that a luminosity changes from ** tacitly when inside of a picture 
by which a person's face is photoed is investigated to a top and bottom 
direction of a picture. 
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A means to detect a field across which it faces in that it changes to 
anc } 

A means by which a shape facility of this field to this field and the 
luminosity feature in a field take out a field near a person's eyes as an 
eye candidate area, 

A means by which a shape facility of a described area to a described area 
and the luminosity feature in a field take out a field near a person's 
mouth as a mouth candidate area, A means which takes out combination 
which has succeeded in arrangement appropriate for [ combine one 
arbitrarily, without overlapping and ] a person's face from two from the 
above-mentioned eye candidate area, and the above-mentioned mouth 
candidate area as a face-candidates field, and a means to detect a 
person's face by comparing a standard face with the above-mentioned 
face-candidates field. 



[Function]The example of an inputted image is shown in drawing 3 (a). It 
is assumed that the face near for human being's transverse planes is 
photoed in a picture. 

[0023]a face — a person — although it is [ each ] various, the shape of 
eyes, a nose, a mouth, and a face, its color, etc. have very strong 
restriction on structure. Especially in this invention, eyes and a mouth 
are observed from this. The input picture signal 51 is cut in the cutting 
plane a52 or the cutting plane b53, and the typical luminance change is 
shown in dra wing 4 . Thus, although eyes and the mouth can carry out 
easy distinction because it is dark compared with the circumference, a 
threshold cannot be said to be a stout algorithm to lighting change by 
giving like the threshold signal 58 like conventional technology. 
[0024]Then, it stops detecting a dark field, the luminosity in dark field 
neighborhood, i.e., drawing 4 , will detect the luminosity minimum point 

inflection a60 which becomes large and goes, and the field across which it 
faces by these two points will be detected. Quadratic differential is 
carried out and primary and the conditions from which the numerals 
become 0 and +, respectively can realize a picture to detect a luminosity 
minimum point. A picture is realizable similarly [ detecting the luminosity 
point of inflection ] by primary and the conditions from which quadratic 
differential is carried out and numerals are set to + and 0, respectively. 
[0025]Using the field across which it faces instead of similarly dark field 
detection in the luminosity point of inflection b61 in drawing 4 which 



[0022] 
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luminosity becomes small locally and goes, and the luminosity minimum 
point 59 which becomes the smallest can also acquire the same effect, 
However, since both sides become the detection processing by the 
conditions that quadratic differential is weak in the noise 0, the field 
across which it faces in the above-mentioned luminosity point of 
inflection a60 and the above-mentioned luminosity point of inflection b61 
is not effective. 

[0026]Described area detection processing can be easily detected, 
without using a threshold, if conventional technology "edge area 
detecting method and device'", for example, JP,5-189566,A, is used. 
Since the dark field is only cut off compared with the circumference, it is 
effective to the white races or general yellow-skinned races. 
[0027]The detected field turns into a candidate area of eyes or a mouth. 
The candidate of eyes and the candidate of a mouth are selected out of 
this "candidate area. Since eyes and a mouth have succeeded in 
characteristic shape and texture, respectively, it asks for each candidate 
area of eyes and a mouth by establishing a loose threshold to the texture 
information in width, height, its ratio, and a field. Although used 
combining the edge point that luminosity changes, in the field detection 
processing which this invention uses, Generally, since the field which ' 
originates in unnecessary natural objects other than eyes and a mouth 
since the edge detection process is sensitive to a noise has accomplished 
complicated shape, it can reduce these fields effectively. 
[0028]From comprising arrangement with one usually unique two eyes 
and a mouth, a person's face is taken out without overlapping one from 
two from the already obtained eye candidate area, and a mouth candidate 
area, investigates arrangement, and checks appropriate for a face. 
[0029]It becomes possible to start a face out of a picture by carrying out 
based on face candidates' eyes and the position of a mouth which were 
acquired, starting a face-candidates picture like drawing 3 (b) from an 

beforehand as shown in the figure (c). 
[0030] 

[Example]One example of this invention is described using drawing 1 
below. 

[0031]There is a picture containing the face of a person as showed 
drawing 3 (a), and this is made into the input picture signal 31. The face 
sensing device 20 which is this invention comprises three processing 
units, the field sensing device 21 which starts the component of a face as 
a field, the lace-candidates sensing device 22 which outputs the 
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candidate of a face from face component area size or physical 
relationship, and the face determining device 23 which questions face 
candidates in detail and judges whether it is a face. 
[0032]The primary differential means 1 performs primary differential in 
the direction of [ lower ] for said input picture signal 31 from on on a 
picture, and the primary differential signal 32 is outputted. Binarization 
of this primary differential signal is carried out by 0 by the binarization 
means 2, and the primary differential binarization signal 33 is outputted. 
The quadratic differential means 3 performs quadratic differential for said 
input picture signal, and the quadratic differential signal 34 is outputted, 
Binarization of this quadratic differential signal is carried out by 0 by the 
binarization means 4, and the quadratic differential binarization signal 35 
is outputted. 

[0033]By the logical sum means 5, the logical sum of said primary 
differential binarization signal 33 and said quadratic differential 
binarization signal 35 is taken, and the first candidate signal 36 of eyes 
and a mouth is outputted. In this first candidate signal of eyes and a 
mouth, a connected area to each accomplished field by the connected 
area feature quantity calculation means 6 Area, A centroid position, the 
length length and beside a field, and said input picture signal are also 
considered as an input, and the characteristic quantity of the field of an 
average and distribution of the luminosity in a field is outputted as the 
field characteristic quantity signal 37 to each field. 

[0034]The second candidate judging means 7 of eyes which considers this 
field characteristic quantity signal as an input investigates the area of 
each field, the length length and beside a field, and a luminance average 
and distribution, and this field judges the field appropriate for eyes, and 
outputs as the second candidate signal 38 of eyes together with the 
characteristic quantity of a field. The second candidate judging means 8 
of a mouth which considers said field characteristic quantity signal as an 
4nput^im-ikr4y-inv^ — 
beside a field, and a luminance average and distribution, and this field 
judges the field appropriate for a mouth, and outputs as the second 
candidate signal 39 of a mouth together with the characteristic quantity 
of a field. 

[0035]By the face-candidates judging means 9, an eye candidate area 
from said second candidate signal of eyes Two. A mouth candidate area is 
chosen from said second candidate signal of a mouth so that one and no 
fields may overlap mutually, the group of the candidate who investigates 
the centroid position of each field and is doing arrangement appropriate 
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for a face is investigated about all the combination, and it outputs as the 
face-candidates signal 40, 

[0036]On the basis of the centroid position of the candidate area of the 
eye of the right and left in this face-candidates signal, the candidate area 
where a face exists is started using affine transformation by the face- 
candidates region image logging means 10, and it outputs as the face- 
candidates picture signal 41 as shown in drawing 3 (b). Compute the 
distance between face standard patterns as indicated to be these face- 
candidates picture signals to drawing 3 (c), and if smaller than a certain 
threshold, It comprises the face judging means 1 1 which judges that 
human being's face is photoed by the place where said input picture 
signal corresponds, and outputs the position, size, and angle in which a 
face exists as the face signal 42. 

[0037]The above-mentioned face determining device 23 described in the 
example is mentioned as easiest example. It is possible to apply various 
art in which it has been conventionally used as a face sensing device as 
this face determining device, and it is possible to reduce the existence 
ranges of a face effectively with the described area sensing device 21 and 
the above-mentioned face-candidates sensing device 22. If it is under the 
situation where background things other than a face have seldom moved 
into the picture, the same effect can be enough demonstrated only in the 
combination of the described area sensing device 21 and the above- 
mentioned face-candidates sensing device 22. 

[0038]The field sensing device 24 which constitutes one example of this 
invention is explained using drawing 5 . The primary differential means 1 
performs primary differential in the direction of [ lower ] for said input 
picture signal from on on a picture, and the primary differential signal 32 
is outputted. Binarization of this primary differential signal is carried out 
by 0 by the binarization means 2, and the primary differential binarization 
signal 33 is outputted. The logic negative means 12 considers the above- 

-mentioned-primar^^u^^ 

calculates a logic negative, and outputs the primary differential 
binarization logic negative signal 43. The quadratic differential means 3 
performs quadratic differential for said input picture signal, and the 
quadratic differential signal 34 is outputted. Binarization of this quadratic 
differential signal is carried out by 0 by the binarization means 4, and the 
quadratic differential binarization signal 35 is outputted. 
[0039]By the logical sum means 5, the logical sum of said primary 
differential binarization logic negative signal 43 and said quadratic 
differential binarization signal 35 is taken, and the first candidate signal 
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36 of eyes and a mouth is outputted, 

[0040]By replacing the special character region sensing device 24 with 
the field sensing device 21 of drawing 1 , the face sensing device which 
can acquire the same effect can be constituted. If it is under the situation 
where background things other than a face have seldom moved into the 
picture, the same eifect can be enough demonstrated only in the 
combination of the described area sensing device 24 and the above- 
mentioned face-candidates sensing device 22. 
[0041] 

[Effect of the Invention]By this invention, a face sensing device strong 
against change of the size of this face, or parallel, a rotation and change 
of lighting conditions can be provided. 
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* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation 
may not reflect the original precisely. 

2. **** shows the word which can not be translated. 
3, In the drawings, any words are not translated, 



DESCRIPTION OF DRAWINGS 

[Brief Description of the Drawings] 
[Drawing l]It is a lineblock diagram of one example of the face sensing 
device which is this invention. 

[Drawing_2]The face sensing devices which are conventional technology 
are six candidates who use on the way. (a) It is a candidate when -(b) - 

(c) has not worn glasses, and (a) is [ (b) - (c) ] correctly mistaken in eyes 
and eyebrows, (d) It is a candidate when -(e) - (f) has worn glasses, and 

(d) is the example for which (e) - (f) mistook eyes and eyebrows 
correctly. 

[Drawing 3j(a) is an example of an inputted image, (b) is the picture 
which obtained it from the inputted image (a) by carrying out affine 
transformation of the face area, (c) is a typical person's face picture. 
[Drawing 4]It is typical behavior of the luminance signal which crosses 
eyes and a mouth. 

[Drawing 5]It is an example of the field sensing device which constitutes 
the face sensing device which is this invention, 
[Description of Notations] 

I Primary differential means 

-^■inarizatiorrTTieans : 

3 Quadratic differential means 

4 Binarization means 

5 Logical sum means 

6 Connected area feature quantity calculation means 

7 The second candidate judging means of eyes 

8 The second candidate judging means of a mouth 

9 Face-candidates judging means 

10 Face-candidates region image logging means 

I I Face judging means 
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12 Logic negative means 

20 Face sensing device 

21 Field sensing device 

22 Face-candidates sensing device 

23 Face determining device 

24 Field sensing device 

31 Input picture signal 

32 Primary, differential signal 

33 Primary differential binarization signal 

34 Quadratic differential signal 

35 Quadratic differential binarization signal 

36 The first candidate signal of eyes and a mouth 

37 Field characteristic quantity signal 

38 The second candidate signal of eyes 

39 The second candidate signal of a mouth 

40 Face-candidates signal 

41 Face-candidates picture signal 

42 Face signal 

43 Primary differential binarization logic negative signal 

51 Input picture signal 

52 Picture cutout line a 

53 Picture cutout line b 

54 Luminance signal 

55 Skin field a 

56 Eyes and a mouth field 

57 Skin field b 

58 Threshold signal 

59 Luminosity minimum point 

60 Luminosity point of inflection a 

61 Luminosity point of inflection b 
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* NOTICES * 

JPO and INPIT are not responsible for any 
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DRAWINGS 



[Drawing Jj 
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[Drawing 21 
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